Exploring the evolution of complexity in signaling networks

Signaling networks are exemplified by systems as diverse as biological cells, economic markets, and the Web. After a discussion of some general characteristics of signaling networks, this paper explores the adaptive evolution of complexity in a simple model of a signaling network. The paper closes with a discussion of broader questions concerning the evolution signaling networks. You are immersed in a world that has two resources distributed in patches, call them “shelter” and “food”, and your needs are determined by the levels of two internal reservoirs, one for each resource. The reservoirs are depleted at a constant rate to keep your system running, so that a low reservoir level implies high need for that resource. At each instant, you can execute one of six simple actions: rest, approach, flee, turn right 45 degrees, turn left 45 degrees, consume. You can increase the level of a reservoir only by giving an appropriate response when the resource is present: “rest” when “shelter” is present, “consume” when “food” is present. Your information about the world is supplied by a “vision cone” that indicates resource locations relative to a line of sight: resource at present location, or straight ahead, or within an angle of 45 degrees to the left or right. Question: Is there a simple adaptive algorithm that can, on the basis of experience, discover action sequences (e.g. “turn until food is visible”, “approach”, “consume”) that exploit opportunities for filling the reservoirs? Though the format seems one of animal cognition, the question applies equally to other systems requiring coordinated responses, such as ecosystems, and the Web, and signaling networks in biological cells. In this broader context, we come to more difficult questions: Are there “general purpose” adaptive algorithms that can discover good action sequences over a wide variety of environments, without prior tuning for each environment? If so, what kinds of ontogeny and phylogeny should we expect to see as the adaptive algorithms modify the system? The objective of this paper is to explore these questions, starting from a simple computer-based model of the “cognitive world” just described. To answer these questions, we must first describe the “cognitive” repertoire of the adaptive agent -the system doing the adapting. Here I’ll restrict the repertoire to a familiar set of possibilities: sets of IF(condition satisfied)/THEN(action) rules. Such rules, in the simplest case, implement the stimulus-response repertoires of classical behavioral psychology. However, we can extend this repertoire considerably by defining the conditions and actions in terms of messages. That is, the agent is given a message-processing repertoire: The detectors (e.g., the vision cone) produce messages, the effectors (e.g., the elementary actions) are activated by messages, and internal processing (e.g., internal feedback and computation) is accomplished by message circulation and transformation. Because there is a considerable literature on such systems, called classifier systems [Lanzi et al. 2000], this paper only presents details directly relevant to the questions being asked. Classifier systems are computation-universal, in the sense that any program that can be written for a general-purpose computer can be executed by an apropos collection of these message-processing rules. This means that any signaling network that can be modeled by a computer simulation can also be modeled by a classifier system. Using a classifier system to define repertoire refines the earlier question to: What kinds of adaptive algorithm can discover useful sequences of messageprocessing rules? Of course, the agent can accomplish the task by simply trying rules at random, gradually collecting those that “work”, but such trail-and-error algorithms are neither interesting nor feasible. Under random trials it takes an unreasonably long time to find even short rule sequences. Is there something better? Can evolutionary processes produce useful sequences in feasible times? Classifier systems are designed for “on-line” modification by genetic algorithms and other adaptive algorithms. This feature opens the possibility of computerbased observation of adaptive changes in simple versions of signaling networks. In particular, simple classifier system models have the possibility of mimicking aspects of the ontogeny and evolution of bio-circuits -the complex signaling networks of molecular biology. The paper begins (section 1) with a general description of complex signaling networks, then uses this description as a guide to present (section 2) an exploratory computer-based model of an adaptive agent in the two-resource world described in the first paragraph. As already suggested, the agent will be implemented (section 3) as a classifier system that evolves as the agent accrues experience. Execution of this model demonstrates (section 4) the adaptive evolution of a resource-seeking signaling network, going from a single rule that produces a random walk to sequences of rules that provide compatible resource-seeking actions. Though the model is quite simple, the underlying program is written to provide easy extensions to much more complex worlds. In particular, the adaptive mechanisms apply without change to the full panoply of agents that can be modeled by classifier systems. The paper closes (section 5) with a discussion of the implications of the model for broader questions about the evolution of complex signaling networks. 1. Complex signaling networks. Complex signaling networks abound, integrating systems as widely different as biological cells and the Web. Despite substantial differences in implementation, complex signaling networks share important characteristics. Chief among these are: (i) Parallelism and coordination. Complex signaling networks, by definition, consist of large numbers of “transmitter/receiver” nodes that send and receive signals. The networks of interest here involve massive simultaneity: many nodes act at the same time, producing large numbers of simultaneous signals. Biocircuits, for example, typically use proteins as signals. These proteins operate in reaction cascades and cycles, providing positive and negative feedback to other cascades and cycles. A biological cell has large numbers of active proteins and their interactions must be tightly coordinated if the cell is to continue to function. Indeed, in all the networks of interest, coordination is a major problem. Each signal must go to appropriate destinations, and it must be appropriately interpreted at those destinations. (ii) Conditional action. In complex signaling networks, nodes only act when they receive an appropriate signal. That is, they have the IF/THEN structure discussed earlier: IF [an apropos signal is present] THEN [act]. The act may itself be a signal, allowing quite complicated feedbacks, or the act may be an overt action such as such as shutting off a mechanism or binding to some site. Interlocking sequences of message-processing rules become programs that are executed in parallel, with all that implies for flexibility and breadth of repertoire. (iii) Modularity. In a sense, a complex signaling network is automatically modular, the nodes being the modules. But that is not what is meant here. If we look to the rules associated with the nodes, it is unlikely that the system can handle a broad range of situations by having one rule for each distinct situation. A rule for reacting to “a red Saab by the side of the road with a flat tire” has many elements, or building blocks, in common with a rule for “a blue Chevy stalled at the intersection”. It is better if the agent can activate a set of rules that react to the elements of the situation. The foregoing situations can be handled easily by combining rules dealing with “car”, “roadside”, “flat tire”, “stalled”, and the like. Reaction cycles that serve as building blocks are a common feature of bio-circuits. For example, the Krebs cycle of eight proteins is used by almost all aerobic organisms . The agent, by simultaneously activating a set of building block rules, can react to a broad range of novel situations, making combinatorics work for the system instead of against it. Also, because appropriate building blocks are used frequently in a wide range of situations, they are tested and confirmed at a high rate. When tested building blocks work in a parallel, coordinated fashion they provide great flexibility, but they force the coordination problem discussed in (i). (iv) Adaptation and evolution. Complex signaling networks change over time. Many of these changes are more than random variations, they are adaptations that improve performance. The performance itself is the result of an intricate skein of interactions extended over space and time. Most of the interactions are distant, in both space and time, from the direct causes of changes in performance. As a result, there is a considerable problem in determining which interactions were responsible for the changes. This problem is often called the credit assignment problem. The play of a game of strategy, such as checkers or chess, provides a useful metaphor for understanding the need for credit assignment: After a long sequence of moves, the player receives notification of a "win" or a "loss" and, perhaps, an indication of the size of the win or loss. There is little information about which moves along the way were critical to that performance. The problem, then, is to determine which moves might be useful in future games. Similarly, in a biological cell, the “reward” of reproduction results from the interactions of hundreds to thousands of signaling proteins over hours or days. The general question is: How does a signaling network allocate credit for desirable outcomes back to the responsible nodes (rules)? There is a mitigating factor that is helpful in resolving this problem: The environments in which signaling networks operate do exhibit perpetual novelty, as does a complicated game like chess, but there are repeating sub-patterns in those environments. In chess these repeating sub-patterns have names like “fork”, “pin”, “gambit”, and so on. In the environment described at the outset the patches constitute repeating elements that can be exploited. In general, such sub-patterns can be exploited by particular arrangements of the signaling nodes, the modules alluded to in (iii). Close attention to the origins and ontogeny of modules used by a signaling network offers vital clues to its organization and performance in response to different environments. It is the thesis of this paper that exploratory models built around these shared characteristics can give insights into the operation and evolution of natural and artificial signaling networks. 2. Signaling networks implemented as classifier systems. This section starts with a definition of classifiers (section 2.1), then goes on to systems of such rules (section 2.2), and concludes with the description of an adaptive agent defined with the help of a classifier system (section 2.3).